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RESUME 

La  quantity  d ' information  observable  dans  une  photographie 
brouill4e  peut  etre  augmentee  si  1' image  est  trait^e  dans  un  calculateur 
num£rique.  On  deer  it  une  procedure  complete  (qui  est  basee  sur  le 
filtrage  de  type  Wiener)  pour  la  reconstitution  des  photographies  de 
haute  resolution  qui  sont  deter iorees  par  plusieurs  sortes  de  brouillage 
cortinun.  On  considere  en  detail  les  aspects  pratiques  d'une 
reconstitution  acceptable.  On  ddcrit  les  limitations  du  filtre  de 
Wiener,  les  caract4r istiques  du  film  photographique  et  des  methodes 
d 'identification  des  brouillages  par  l'analyse  des  photographies 
brouillees.  Des  photographies  de  haute  resolution  defocalisees'  et 
deteriorees  par  le  mouve-.ient  qui  est  introduit  directement  dans  la 
camera  sont  reconstitutes  pour  illustrer  les  techniques.  (NC) 


ABSTRACT 

The  amount  of  information  that  a human  observer  can  obtain  from 
a blurred  photograph  can  be  increased  by  post-processing  the  image  in  a 
digital  computer.  A complete  procedure  for  restoring  (by  Wiener 
filtering)  high-resolution  photographs  degraded  by  several  common  types 
of  blurs  is  described.  The  practical  aspects  of  obtaining  an  acceptable 
restoration  are  considered  in  detail.  These  include  the  limitations  of 
the  Wiener  filter,  the  characteristics  of  photographic  film  and  methods 
to  identify  the  blur  by  analysing  the  blurred  image  using  only  limited 
a priori  knowledge.  High-resolution  photographs  degraded  by  motion  and 
focus  blurs  introduced  directly  in  a camera  are  restored  to  illustrate 
the  techniques.  (U) 
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iterative.  On  the  other  hand,  linear  methods  are  more  easily  analysed 
and  efficient  algorithms  such  as  the  fast  Fourier  transform  (FFT)  (refs. 
17,  26  and  36)  can  be  used  for  certain  types  of  linear  restoration. 

We  describe  here  a procedure  for  restoring  blurred  photographs 
based  on  one  of  the  linear  techniques  - Wiener  filtering  - implemented 
in  the  spatial-frequency  domain  using  the  FFT  algorithm.  Practical 
aspects  of  obtaining  an  acceptable  restoration  are  emphasized.  In  many 
cases,  such  filtering  can  produce  useful  restorations  of  high-resolution 
photographs  degraded  by  several  common  types  of  blurs.  It  can  be 
implemented  on  a medium-scale  digital  computer  or  on  a minicomputer  with 
a special-purpose  hardware  processor. 

Characteristics  of  photographic  film,  including  granularity  and 
nonlinearity,  are  first  described  and  the  criteria  for  digitizing  the 
film  are  given.  The  Wiener  filter  is  related  to  restoration  of  blurred 
photographs.  Problems  associated  with  film  nonlinearity  and  noise, 
spatial  variance  of  the  blur  and  with  edge  effects  are  considered. 
Examples  which  demonstrate  the  removal  of  simulated  out-of-focus, 
linear-motion  and  long-exposure  atmospheric  turbulence  blurs  are  given. 
Photographs  containing  out-of-focus  and  motion  blurs  introduced  directly 
in  the  camera  are  analysed  to  determine  the  nature  of  the  blur  and  are 
restored.  Two  methods  for  determining  the  type  of  the  blur  (e.g.  focus) 
and  its  extent  (e.g.  diameter  of  blur  circle)  directly  from  the  blurred 
image  are  described. 
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Film  noise  can  also  be  expressed  as  an  RMS  fluctuation  in  transmittance 
around  a mean  transmittance  T , S^(T,A).  For  a density  well  above  the 
fog  level  of  the  film  (i.e.  the  density  corresponding  to  zero  exposure), 
the  density  and  transmittance  fluctuations  are  related  (ref.  40): 


St(T,A) 


T Sd(D,A) 
0.4343 


where  T=10  . If  the  film  noise  (S_)  is  known  at  one  transmittance  T , 

r o 

it  can  be  calculated  at  another  transmittance  T (ref.  40): 

vt'a>  - vva>  (3) 

Substituting  Eqs.  1 and  2 into  Eg.  3 gives  expressions  for  the  film 
noise  (ST  and  SD)  in  terms  of  the  known  film  noise  and  the 

area  A of  the  aperture  used  to  digitize  the  film: 

s.(t,a)  - w,  (4) 

0.4343  Y A V 1-T 


vd'a» 

\ A \ T(W0) 

where  T*10~D  and  T »1B  °. 

o 


We  define  the  signal-to-noise  ratio  of  a digitized 


transmittance  value  T as: 
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SNR(T)  = 


T S^A)  St(T,A) 


Using  Eg.  4,  one  can  write: 


SNR(T)  = 


>.4343  Ja"  / 

)<W  >Ao\; 


1-T 


TqT(I-T) 


(6) 


(7) 


The  signal-to-noise  ratio  given  by  Eg.  7 is  plotted  as  a function  of 

1/2 

transmittance  in  Fig.  1.  The  scale  on  the  left  has  been  divided  by  A ' 
(in  micrometers)  and  has  been  multiplied  by  SD(Do,AQ)  (in  density  units) 
to  give  a scale  vrtiich  applies  for  arbitrary  film  granularity  and 
digitizing  aperture.  The  standard  conditions  Dq=1  and  Aq,  the  area  of  a 
circular  aperture  with  a 48-um  diameter,  were  assumed. 


This  graph  allows  one  to  estimate  the  signal-to-noise  ratio  of 
a film  with  known  granularity  that  was  digitized  with  a given  aperture. 
As  will  be  described  in  Sec.  3.0,  a value  for  the  signal-to-noise  ratio 
must  be  specified  in  the  filter  used  to  restore  a blurred  image. 
Secondly,  if  the  signal-to-noise  ratio  of  the  film  is  known,  the  maximum 
amount  of  restoration  possible  for  a given  blurred  image  can  be 
estimated  before  the  restoration  is  actually  atterrpted. 


As  an  exanple  illustrating  the  use  of  this  graph,  a photograph 
recorded  on  a Panatomic-X  film  was  digitized  with  a 100-  ym  square 
aperture.  Ihe  RMS  variances  within  6 small  uniformly  exposed  areas  of 
the  film  were  calculated.  Using  Eq.  5,  we  converted  each  RMS  variance 
to  the  above  standard  values  of  Dq  and  Aq  and  we  averaged  the  6 values 
to  obtain  the  result  Sj^D^A^  * 0.020.  Ihe  left-hand  scale  on  the 
graph  was  divided  by  this  film  granularity  and  multiplied  by  the  square 


Transmittance  { T ) 


FIGURE  1 


The  signal-to-noise  ratio  of  silver  halide  film  as  a 
function  of  film  transmittance 
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root  of  the  area  of  the  digitizing  aperture  A ' =100  urn.  Ttie  resulting 
s ignal - to-no i se  ratio,  expressed  in  decibels,  is  shown  on  the  right-hand 
scale  of  the  graph.  Note  that  a factor  of  2 decrease  in  the  diameter  of 
the  digitizing  aperture,  or  a factor  of  2 increase  in  the  film 
granularity,  decreases  the  signal-to-noise  ratio  by  3 dB  at  any 
transmittance. 

The  signal-to-noise  ratio  of  the  film  depends  significantly  on 
the  density.  In  the  present  example,  regions  of  the  film  with  a 
transmittance  of  0.5  (D=0.30)  have  a 25-db  signal-to-noise  ratio  while 
regions  with  low  average  transmittance  (T=0.05,  D=1.30)  or  with  high 
average  transmittance  (T=0.95,  D=0.022)  have  a signal-to-noise  ratio 
increased  to  29  dB.  This  means  that  (neglecting  the  effects  of  film 
nonlinearity) , it  will  be  more  difficult  to  restore  regions  with  an 
average  transmittance  near  0.5  than  regions  with  a higher  or  a lower 
average  transmittance  (Sec.  3.0). 

The  above  noise  calculations  are  based  on  the  probability  of 
overlap  of  the  developed  silver  grains.  At  low  exposure  levels 
additional  noise  due  to  variation  in  the  number  of  silver  grains 
activated  by  the  light  photons  will  contribute.  This  (along  with  other 
possible  noise  sources)  will  decrease  the  effective  signal-to-noise 
ratio  below  that  predicted  by  Eg.  7. 

2.2  Film  Transfer  Function 


An  exposure  g(x,y)  will  produce  a film  density: 

D(x,y)  = y log[g(x,y)l  - Df  (8) 
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where  y is  the  film  gamma  (y  is  positive  for  negative  film  and  negative 
for  positive  film)  and  is  the  fog  level  of  the  film.  One  obtains  the 
transmittance  through  the  film  T(x,y)  by  exponentiating  the  density: 


D(x,y)  -Df  y 

T(x,y)  =10  =10  [g(xry)] 


(9) 


It  is  evident  from  Eq.  9 that  the  film  transmittance  is  linearly  related 
to  the  original  exposure  only  if  y=l. 

In  addition  to  a non-unity  y,  a number  of  other  effects  can 
produce  a nonlinear  relationship  between  film  exposure  and 
transmittance.  The  response  given  by  Eq.  8 does  not  hold  for  very  high 
or  very  low  exposure  levels.  Adjacency  effects  (ref.  42)  during 
development  tend  to  sharpen  the  boundary  between  two  areas  of  different 
exposure.  This  is  caused  by  a redistribution  of  developer  and  reaction 
products  across  the  boundary  which  changes  the  rate  of  development. 
Light  scattering  within  the  film  emulsion  during  exposure  (halation) 
will  tend  to  spread  out  a sharp  boundary  (ref.  42) . The  effect  of 
attempting  to  restore  a blurred  image  after  modification  by  the 
nonlinearities  present  in  the  transfer  function  of  the  film  is 
considered  in  Sec.  3.2. 

2.3  Film  Digitization 

The  first  step  in  the  present  restoration  procedure  is  to 
digitize  the  film  image  (convert  it  to  digital  computer  format) . This 
is  performed  by  sampling  the  film  image  point  by  point  using  a given 
aperture  and  sampling  interval  (usually  the  aperture  diameter  is  equal 
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to  the  sampling  interval) . For  each  point  (image  element) , the  film 
transmittance  or  density  is  measured  and  quantized  into  a finite  number 
of  levels.  The  quantization  in  space  and  density  (or  transmittance) 
must  be  fine  enough  to  preserve  all  the  information  of  interest  in  the 
film  image  and  to  prevent  aliasing.  Ihe  present  section  discusses 
criteria  for  performing  correct  spatial  and  density  quantization. 

(Xiantization  of  film  density  rather  than  film  transmittance  is 
often  preferred  because,  in  most  cases,  the  resulting  values  are  more 
uniformly  distributed  (ref.  29).  As  is  evident  from  Eq.  3,  the  noise 
S.J,  is  maximum  at  transmittance  T=0.5  (D=0.30).  Following  Billingsley 
(ref.  40),  we  choose  the  size  of  the  density  steps  aD  equal  to  3 times 
the  noise  SD  at  this  density: 


AD  = 3 Sd(D=0.3,A) 


(10) 


Using  Eq.  5,  the  density  step  size  can  be  written  in  terms  of  the  known 
film  noise  sD(D0rAQ)  a™3  the  area  of  the  digitizing  aperture  A: 


AD  = 3 S0(Do,Ao) 


(ID 


For  example,  using  a 100  um  square  aperture  to  digitize  the  Panatomic-X 
film  described  in  Sec.  2.1  (Sd(D0,A0)  = 0.02  where  Dq  =1  and  AQ  is  the 
area  of  a circular  aperture  with  48-um  diameter)  requires  a density  step 
size  AD=0.0085.  If  a 3D  density  range  is  to  be  digitized,  then  at  least 
3/AD-354  uniformly  spaced  density  levels  are  required. 
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In  performing  the  spatial  quantization,  one  has  to  assume  that 
the  film  image  band-limited  to  an  upper  spatial  frequency  WQ.  Hie 
sampling  interval  must  then  be  1/2WQ  or  less  to  preserve  the  highest 
spatial  frequency.  Band-limiting  of  the  film  image  can  result  from  the 
spatial-frequency  content  of  the  original  scene  which  was  photographed, 
or  it  can  correspond  to  the  cutoff  of  the  camera  lens  or  photographic 
film.  (For  example,  taking  the  cutoff  frequency  of  Kodak  Plus-X  film  to 
be  50  cycles/mm  (ref.  42)  requires  a sampling  interval  of  10  urn  or 
less.) 
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3.0  WIENER  FILTER  RESTORATION 


3.1  Restoration  Filter 

Assume  that  a degraded  image  has  been  produced  in  the  following 
manner.  First,  a blur  defined  by  a spatially  invariant  point-spread 
function  h(x,y)  is  convolved  with  the  "ideal"  image  f (x,y).  The 

v*  C 

point-spread  function  (PSF)  is  the  result  obtained  when  the  blur  is 
applied  to  a single  point.  Spatial  invariance  means  that  the  blur  is 
the  same  throughout  the  image.  Random  noise  n(x,y)  with  zero  mean  is 
then  added  to  produce  the  degraded  image  gc(x,y)  available  for 
restoration: 

oo  ao 

9c(x*y)  sJJhc(x'  »y')  fc(x-x‘ ,y-y‘)  dx'dy'  + n(x,y)  (12) 

•OO  -oo 

The  model  assumed  for  the  degradation  is  shown  schematically  in  the 
upper  part  of  Fig.  2. 


Consider  the  discrete  function  g(x,y)  obtained  by  digitizing  a 
region  of  the  continuous  image  gc(x,y)  over  an  N-  by  N-element  matrix. 


Continuous  spatial  functions  have  the  subscript  "c" 


while  the 


corresponding  sampled  functions  have  no  subscript.  Within  a 
normalization  factor,  the  2-dimensional  discrete  Fourier  transform  of 
g(x,y)  is  given  by  (ref.  26): 


N-l  N-l 


G(u,v) 


g(x,y)  exp[-2wi(ux+vy)/N) 


xao  yio 


/ 


UNCLASSIFIED 

12 

DEGRADATION 


Ideal 

Image  <>■ 
f(x.y) 


Convolution 
with  h(a./3) 


Noise 

n(x.y) 


Degraded 

Image 

g(*.y) 


RESTORATION 


Degraded 

Image 

9(*.y) 


Restored 

Image 

r(x.y) 


\ 

FIGURE  2 Block  diagrams  of  the  Wiener-filter  restoration  procedure 
(lower)  ana  of  the  model  assumed  for  the  degraded  image 
(upper) 
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and  the  inverse  transform  by: 

N«l  N-l 

g(x,y)  = EE  G(u,v)  exp[2*i (ux+vy)/^]  (H) 

xio  y»o 

In  the  spatial-frequency  domain,  one  may  perform  the 
convolution  given  in  Eq.  12  by  multiplying  the  discrete  Fourier 
transform  of  the  ideal  image  by  the  discrete  Fourier  transform  of  the 
PSF  of  the  blur: 

G (u,v)  = F (u,v)  H(u,v)  + N (u,v)  (15) 

The  discrete  Fourier  transform  of  the  PSF  of  the  blur  H(u,v)  is 
henceforth  referred  to  as  the  optical  transfer  function  (OTF)  of  the 
blur. 

The  restoration  problem  consists  in  estimating  f(x,y)  given 
g(x,y).  One  objective  criterion  for  performing  the  restoration  is  to 
minimize  the  mean-square  error  between  the  ideal  image  and  the  restored 
image  r(x,y).  Under  the  above  assumptions  the  linear  shift-invariant 
filter  which  minimizes  the  mean-square  error  has  a frequency  response 
(refs.  5 and  17): 

H*(u,v) 

Q(u,v) (16) 

|H(u,v) | + ♦n(ufv)/*f(u,v) 

where  H(u,v)  is  the  OTF  (the  asterisk  refers  to  the  complex 
conjugate) , 

♦n(u,v)  is  the  power  spectrum  of  the  noise,  and 
♦f(u,v)  is  the  power  spectrin  of  the  ideal  image. 
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This  is  generally  referred  to  as  the  Wiener  restoration  filter  (refs. 
1-3)  although  it  was  first  derived  in  the  context  of  image  restoration 
by  Helstrom  (ref.  11).  Other  restoration  criteria  have  led  to  filters 
with  a similar  form  (e.g.  refs.  1-4  and  10). 


i 

The  restored 

image  r(x,y)  is 

obtained 

by  calculating 

the 

discrete  Fourier  transform  of  g(x,y) 

(using 

a 2-dimensional 

FFT 

algorithm  as  described 

in  refs.  17,  45 

and  47) , 

multiplying  by 

the 

Wiener  filter  (Eq.  16): 

R(u,v)  = G(u,v)  Q(u,v)  (17) 


and  calculating  the  inverse  discrete  Fourier  transform.  The  restoration 
procedure  is  shown  schematically  in  the  lower  part  of  Fig.  2.  Problems 
associated  with  application  of  this  method  to  restore  blurred 
photographs  are  discussed  in  Secs.  3.2,  3.3  and  3.4. 


3.2  Application  to  Blurred  Photographs 

The  Wiener  filter  given  by  Eq.  16  assumes  that  convolving  a 
soatially  invar iant  PSF  with  the  ideal  image  and  adding  random  noise 
produces  the  image  to  be  restored.  Most  image  blurs  of  interest  are  to 
some  extent  spatially  variant  and  film  noise  is  not  additive  with 
exposure  (Sec.  2.1).  Lastly,  the  image  available  for  restoration  is  not 
the  exposure  but  the  exposure  after  modification  by  the  nonlinear 
transfer  function  of  the  film  (Sec.  2.2).  The  effect  that  these 
deviations  from  the  assumed  model  can  have  on  the  restoration  of  blurred 
photographs  is  considered  next. 
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Spatial  invariance  of  the  degradation  means  that  the  same  PSF 

* 

is  convolved  with  all  regions  of  the  image.  Some  blurs  are  spatially 
invariant,  others  are  not.  For  example,  if  all  regions  of  the  scene 
being  photographed  are  located  at  the  same  range  from  the  lens  of  an 
out-of- focus  camera,  then  the  blur  will  be  the  same  throughout  the  image 
(i.e.  spatially  invariant).  Oi  the  other  hand,  the  images  of  objects 
located  at  different  ranges  from  an  out-of-focus  camera  will  be  blurred 
by  different  amounts  (i.e.  the  blur  will  be  spatially  variant). 

If,  for  example,  an  image  containing  spatially  variant  focus 
blur  is  restored  by  Wiener  filtering  using  one  assumed  value  of 
blur-circle  diameter,  only  regions  of  the  image  which  are  blurred  by 
approximately  that  amount  will  be  correctly  restored  - other  regions 
will  be  either  over  or  under  restored.  The  amount  of  spatial  variance 
that  can  be  present  before  the  Wiener  filter  no  longer  produces  an 
acceptable  restoration  depends  on  a number  of  factors.  One  factor  is 
the  amount  of  restoration  sought  (a  large  amount  of  restoration,  which 
is  obtained  when  a small  value  of  ♦pAf  is  specified  in  Eg.  16,  reauires 
a more  accurate  estimate  for  the  PSF) . Other  factors  are  the  type  and 
extent  of  the  blur  (a  blur  with  a large  PSF  reauires  that  the  PSF  be 
known  to  a lower  percent  error  as  shown  in  ref.  18).  Additional 
problems  can  arise  \dien  artifacts  generated  by  incorrect  restoration  of 
one  region  of  an  image  extend  into  correctly  restored  regions.  Examples 
illustrating  acceptable  restorations  of  photographs  containing  blurs 
with  both  large  and  small  amounts  of  spatial  variance  are  given  in  Sec. 
4.2. 
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The  main  source  of  noise  in  photographic  imagery  is  usually 
film  granularity.  This  noise  is  additive  in  film  density  and  therefore 
multiplicative,  not  additive,  in  film  transmittance  (Sec.  2.1).  Whether 
it  is  preferable  to  take  the  correct  blurring  model  and  restore  the  film 
transmittance  image,  or  to  take  the  correct  noise  model  and  restore  the 
density  image  (and  then  exponentiate  to  obtain  the  restored 
transmittance  image)  can  depend  on  the  nature  of  the  particular  image. 
Neither  approach  is  consistent  with  the  model  assumed  in  Ea.  16.  If  the 
image  to  be  restored  were  recorded  on  low-granularity  film,  then 
restoration  of  the  transmittance  image  could  be  preferable.  Here  the 
extent  of  the  possible  restoration  may  not  be  limited  by  film  noise  but 
rather  by  other  effects,  such  as  film  nonlinearity  or  the  accuracy  to 
which  the  PSF  can  be  determined  (or  is  spatially  invariant).  For 
high-granularity  film  where  the  restoration  is  noise-limited, 
restoration  of  the  density  image  could  yield  a better  result. 
Density-domain  restoration  also  has  the  advantage  that  it  constrains  the 
restored  image  to  contain  no  negative  values  as  is  required  for  all 
images  formed  with  incoherent  light.  Examples  comparing  density-  and 
transmittance-domain  restorations  of  blurred  photographs  are  given  in 
Sec.  4.2. 


The  Wiener  filter  assumes  that  the  PSF  of  the  blur  is  convolved 
with  the  original  film  exposure.  Digitizing  the  film  transmittance  or 
density  produces  the  image  to  be  restored.  A nonlinear  relationship 
between  film  exposure  and  the  image  being  restored  (e.g.  due  to  film 
nonlinearity  or  to  restoration  of  the  density  rather  than  transmittance 
image)  can  adversely  affect  the  restoration. 
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Such  nonlinearities  should  have  a significant  effect  only  if 

0 

they  are  appreciable  over  the  area  of  the  PSF  of  the  Wiener  filter.  For 
example,  nonlinearity  due  to  film  gartma  should  be  important  only  in 
regions  of  the  image  where  the  power-law  relationship  between  film 
exposure  and  transmittance  (Eq.  9)  is  not  approximately  linear  over  the 
PSF  of  the  Wiener  filter.  Similarly,  restoration  of  the  density  rather 
than  the  transmittance  image  should  be  acceptable  in  all  regions  where 
the  logarithmic  density  function  can  be  considered  linear  over  the  PSF 
of  the  Wiener  filter.  Gradual  nonlinearities  where  the  above 
small-signal  approximation  is  valid  (e.g.  nonlinearities  due  to  film 
gamma  or  density-domain  restoration)  should  be  less  detrimental  than 
« sharper  nonlinearities  (e.g.  film  saturation). 

It  must  be  remembered  that  the  PSF  of  the  Wiener  restoration 
filter  can  be  considerably  larger  than  the  PSF  of  the  blur.  The  PSFs  of 
the  Wiener  filters  required  to  restore  out-of-focus,  linear-motion  and 
long-exposure  atmospheric  turbulence  blurs  are  shown,  enlarged,  in  Fig. 
3 along  with  the  PSF  of  each  blur.  Each  blur  and  the  corresponding 
restoration  filter  are  to  the  same  scale. 


3.3  Artifact  Generation 


Hie  Wiener  filter  can  introduce  artificial  features  into  the 
restored  image  vhich  are  not  present  in  the  ideal  image  (ref.  43).  This 
is  distinct  from  enhancement  of  noise  by  the  filter  as  artifacts 
generally  have  a well-defined  rather  than  a random  structure.  For 
example,  restoration  of  a blurred  image  using  the  incorrect  OTF  in  the 
Wiener  filter  (Sec.  3.2)  can  under  some  conditions  generate  artifacts. 
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FIGURE  3 
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Point-spread  functions  of  three  common  image  blurs:  (a) 
out-of- focus,  (b)  linear -motion  and  (c)  long-exoosure 
atmospheric  transmission.  The  PSF  s of  the  Wiener  filters 
used  to  restore  the  3 blurs  are  given  in  (d) , (e)  and  (f) 

respectively. 
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Two  other  types  of  artifacts  which  are  often  observed  in  the  restored 
image  are  described  here. 

Ringing  can  take  place  if  the  Wiener  filter  is  applied  to 
defects  introduced  after  the  blurring  has  occurred.  Such  defects  could 
be  caused,  for  example,  by  specks  of  dirt  or  scratches  on  the  film  or  by 
sharp  nonlinearities  (e.g.  due  to  film  saturation).  A defect  with  a 
sufficiently  small  area  will  generate  an  artifact  consisting  of  a 
replica  of  the  PSF  of  the  Wiener  filter.  The  PSFs  of  the  Wiener  filters 
used  to  restore  3 types  of  blur  are  shown,  enlarged,  in  Fig.  3.  If  one 
of  these  "signatures"  is  observed  in  the  restored  imaqe,  it  can  indicate 
the  presence  of  a defect  in  the  original  image.  If  a defect  or  a 
small-area  nonlinearity  can  be  detected  by  inspection  of  the  imaqe 
either  before  or  after  restoration,  then  the  corresponding  artifact  can 
usually  be  reduced  by  appropriate  editing  (e.q.  the  affected  area  of  the 
original  image  can  be  replaced  if  the  surrounding  image  elements  are 
extrapolated) . 

Artifacts  can  also  be  produced  at  the  edges  of  the  restored 
image.  Image  elements  within  a distance  equal  to  the  PSF  of  the  Wiener 
filter  from  the  edges  of  the  original  imaqe  cannot  be  correctly 
restored.  Oscillation  of  the  edges  of  the  image  can  occur  if  the  image 
borders  are  not  properly  treated  before  restoration.  One  way  to 
minimize  this  artifact  is  to  multiply  the  image  to  be  restored  by  a 
window  function  (e.g.  refs.  36  and  44)  to  make  its  edges  qo  smoothly  to 
zero.  Another  approach  is  to  match  the  opposite  edqes  of  the  image  to 
one  another  over  a given  border  region  by  fitting  a low-order  oolynomial 
(ref.  15).  Since  the  present  fast  Fourier  transform  filtering  assumes 
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that  the  opposite  edges  of  the  image  are  adjacent  (e.g.  refs.  26  and 
36),  the  latter  approach  was  used  here.  In  most  cases,  a linear 
interpolation  over  a region  5 image  elements  wide  around  the  border  of 
each  512-  by  512-element  image  was  sufficient  to  reduce  edge  effects  to 
an  acceptable  level  (ref.  45).  This  procedure  was  superior  to  the  use 
of  a bell-cosine  window  function  (ref.  36)  in  its  ability  to  reduce 
oscillation  and  also  required  modification  of  fewer  image  elements. 

3.4  Signal-to-Noise  Ratio  of  Wiener  Filter 

To  optimize  (in  a least  mean-square  error  sense)  the 
restoration  of  a blurred  image  in  the  presence  of  noise,  the  ratio  of 
the  power  spectrum  of  the  noise  to  the  power  spectrum  of  the  ideal  image 
must  be  specified  in  the  Wiener  filter.  In  effect,  this  ratio 
determines  the  amount  of  restoration  performed  by  the  filter.  This 
ratio  is  related  to  the  signal-to-noise  ratio  of  the  film  obtained  in 
Sec.  2.1. 


We  interpret  *f(u,v)  not  as  the  power  spectrum  of  the  ideal 
image  but  rather  as  a weighting  factor  which  represents  the  relative 
importance  of  various  spatial  frequencies  in  the  image  in  conveyinq 
useful  information.  Assume  further  that  it  is  equally  important  to 
restore  all  spatial  frequencies  (up  to  the  Nyquist  frequency) 
independent  of  their  actual  probability  of  occurrence  in  the  ideal  image 
(i.e.  (u,v) =constant) . Since  the  spatial -frequency  content  of  many 
images  falls  off  rapidly  at  high  spatial  frequencies  (e.g.  ref.  6),  this 
will  yield  a restored  image  with  a higher  mean-square  error.  However, 
the  edge  or  small-area  structure  represented  by  the  higher  spatial 


( 
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frequencies  is  often  of  much  interest.  The  assumption  is  that  it  is 
preferable  to  accept  a higher  mean-square  error  (resulting  in  a restored 
image  with  a noisier  appearance)  in  order  to  obtain  additional 
restoration  of  edge  and  small-area  structure. 


Setting  ♦ (u,v)  equal  to  the  power  spec trim  of  the  film  noise 
(which  is  assumed  to  be  uniform  as  described  in  Sec.  2.1)  makes  the 
ratio  (u,v)/*n (u,v)  constant  as  a function  of  spatial  frequency.  As 
an  estimate  for  this  ratio,  we  equate  it  to  the  signal-to-noise  ratio  of 
the  film  given  by  Eq.  7 at  the  average  transmittance  Tav: 


*f(u,V)  = 0.4343  nr  / 1 To 

♦n (u ' v)  Sd(Do'Ao}  1 Ao  \toT3v  ( 1-Tav) 


(18) 


Equation  18  gives  an  estimate  for  the  amount  of  restoration  that  can  be 

achieved  by  the  Wiener  filter.  It  assumes  that  the  restoration  is 

limited  by  film  noise  and  that  the  film  transmittance  values  are 

distributed  within  a narrow  range  about  T . 

av 


Large  values  of  4f/4n  specified  in  the  Wiener  filter  produce 
more  restoration  but  also  more  enhancement  of  noise.  An  example  showing 
how  the  amount  of  restoration  of  an  out-of-focus  photograph  depends  on 
the  value  of  4f/4n  assumed  in  the  Wiener  filter  is  given  in  Fig.  4. 
This  figure  shows  the  original  blurred  image  (upper  left)  after 
restoration  using  8 different  assumed  values  of  ♦f/4n  between  10dB  and 
40  dB.  Almost  no  improvement  is  evident  for  a of  10  dB  while 
increasing  the  ♦f/*n  from  15  dB  to  30  dB  produces  a successively  better 
restoration.  Above  35  dB  there  is  excessive  enhancement  of  film  noise. 
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FIGURE  4 Restoration  as  a function  of  the  value  of  specified  in 

the  Wiener  filter.  A region  of  a photograph  Saken  with  the 
camera  out  of  focus  is  given  in  the  upper  left.  The 
following  7 photographs  show  the  restoration  obtained  as 
the  value  of  $-/<(>  (indicated  in  the  lower-left  of  each 
restored  image!  i§  varied  in  5 dB  steps  from  10  dB  to  40  dB. 
The  same  region  of  the  sign  photographed  with  the  camera  in 
focus  is  given  in  the  lower  right. 
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In  practice,  Eq.  18  may  best  be  interpreted  as  an  initial 

estimate  for  the  value  of  to  be  used  in  the  Wiener  filter  \diich 

can  be  adjusted  to  obtain  the  best  restoration.  In  normal  high-contrast 

photographs,  the  average  transmittance  in  various  reqions  of  the  image 

can  differ  widely.  In  this  case,  no  single  value  of  T will  be  optimum 

av 

for  all  regions  cf  the  image.  For  example,  as  was  explained  in  Sec. 
2.1,  tne  signal-to-noise  ratio  varies  by  4 dB  as  the  film  transmittance 
changes  from  0.5  to  0.95.  Restoration  of  a region  of  a blurred  image 
with  average  transmittance  0.95  then  recruires  a 4 d8  higher  than 

that  of  a region  with  average  transmittance  0.5.  As  illustrated  in  Fiq. 
4,  a 5 dB  difference  in  the  value  of  specified  in  the  Wiener 

filter  can  have  an  appreciable  effect  on  the  quality  of  the  restoration. 

Secondly,  the  restoration  may  not  be  film-noise  limited.  Other 
considerations  such  as  the  presence  of  a spatially  variant  blur  (Sec. 
3.2)  or  the  attempt  to  minimize  artifacts  (e.g.  due  to  film  saturation) 
(Sec.  3.3)  may  make  the  use  of  a value  lower  than  that  predicted  by  Ea. 
18  desirable.  For  lower  values  of  the  Wiener  filter  is  less 

sensitive  to  the  use  of  an  incorrect  OTF  and  artifact  generation  is 
usually  less  severe.  Depending  on  the  nature  of  the  image  to  be 
restored  and  the  information  of  interest  in  it,  the  best  restoration  may 
be  obtained  when  a value  either  higher  or  lower  than  that  predicted  by 
Eq.  18  is  used. 
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4.0  RESTORATION  EXAMPLES 


4.1  Simulated  Blurs 


Assuming  complete  knowledge  of  the  degradation,  the  extent  to 
which  a blurred  photograph  can  be  restored  is  limited  only  by  the  amount 
of  noise  which  is  present.  Simulated  degradations  due  to  an 
out-of-focus  lens  (with  circular  aperture),  linear  motion  and 
long-exposure  atmospheric  transmission  were  applied  to  an  image  as  shown 
in  Figs.  5(a),  (c)  and  (e) . We  introduced  the  blurs  by  calculating  the 
2-dimensional  FFT  of  the  ideal  transmittance  image,  multiplying  by  the 
appropriate  OTF  and  calculating  the  inverse  FPT.  No  noise  other  than 
that  caused  by  computing  errors  was  added. 

The  focus  blur  was  assumed  to  have  a PSF  consisting  of  a 
uniform  circular  disk  of  diameter  d with  an  OTF  (refs.  9 and  13): 

Hx(u,v)  = Jl(l>wd)  (19) 
•»wd 

is  the  Bessel  function  of  the  first  kind  and  first  order, 
The  PSF  of  the  linear-notion  blur  was  a line  of  lenqth  1 
to  the  x-axis  with  an  OTF  (refs.  9 and  13): 

^(UjV)  * sin ( >fl)  (20) 
»fl 

where  f=u  cos (9)  + v sin(0).  For  a long  exposure  (e.g.  10<*  ms),  the  CTF 
of  blurring  caused  by  transmission  through  atmospheric  turbulence  is 
approximately  gauss ian  (ref.  5)  and  was  taken  here  as: 


where  ( . ) 

2 2 2 
and  w *uz+v  . 

at  an  angle  © 


FIGURE  5 Simulated  blurs  due  to  (a)  an  out-of-focus  camera,  (c) 
linear  motion  and  (e)  long-exposure  atmospheric  transmission 
were  applied  to  a photograph.  Each  blurred  image  was  then 
restored  using  a Wiener  filter  as  shown  in  (b) , (d)  and  (f) 
respectively.  Despite  the  large  amount  of  blurring  present 
in  each  case,  the  low  noise  level  allowed  almost  complete 
restoration. 
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H3(u,v)  - exp[-(b2w2/2.155)5/6l  (21) 

2 2 2 

where  b is  the  full  width  at  half  maximum  of  the  PSF  and  w *u  +v  . The 
modulus  of  the  OTF  of  each  of  the  3 blurs  (i.e.  the  modulation  transfer 
function  (MTF) ) is  given  in  Fig.  6.  For  the  focus  blur,  d*0.04  x 
picture  width;  for  the  motion  blur,  1*0.04  x picture  width  and  9*35°; 
and  for  the  atmospheric  blur,  b*0.01  x picture  width. 

Each  blurred  image  was  restored  by  calculating  the  FFT, 
multiplying  by  the  Wiener  filter  (with  H(u,v)  equal  to  the  appropriate 
OTF  and  ♦^/♦n=50  dB)  and  calculating  the  inverse  FFT.  The  restored 
images  are  shown  in  Figs.  5(b),  (d)  and  (f).  Despite  the  large  amount 
of  blurring  present  in  each  case,  the  low  noise  level  permitted  almost 
complete  restoration.  Only  the  information  present  in  each  blurred 
image  was  actually  used  to  obtain  the  restoration. 

4.2  Experimental  Camera  Blurs 

Four  examples  showing  the  restoration  of  experimental  focus  and 
motion  blurs  introduced  directly  in  the  camera  are  now  given.  For  both 
types  of  degradation,  the  amount  of  blurring  varies  with  the  distance 
between  the  camera  lens  and  the  object  being  photographed.  The  blurs 
will  then  be  spatially  invariant  only  if  all  objects  in  the  photograph 
are  located  at  the  same  distance  from  the  camera  lens.  In  addition, 
spatial  invariance  of  the  motion  blur  requires  that  the  motion  be  in  a 
plane  perpendicular  to  the  optical  axis.  The  first  2 examples  show 
restoration  of  spatially  invariant  focus  and  motion  blurs  from 
photographs  of  flat  signs  located  in*  a plane  perpendicular  to  the 


The  Wf  s of  the  (a)  focus,  (b)  linear-motion  and  (c) 
atmospheric  blurs  present  in  the  images  of  Fig.  5.  The 
spatial-frequency  origin  is  at  the  center  of  each  HIF  and 
the  2 axes  span  the  range  between  plus  and  minus  the  Nyquist 
frequencies. 
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optical  axis  where  these  conditions  are  approximately  satisfied.  The 
third  and  fourth  examples  show  restoration  of  photographs  containing 
spatially  variant  motion  blurs. 

Hie  photographs  shown  in  Figs.  7,  12,  14  and  21  were  taken  with 
120  ran  Kodak  Panatomic-X  films.  Hie  density  range  0-2D  of  the  films  was 
digitized  into  256  uniformly  spaced  levels  with  an  Optronics 
rotating-drum  microdensitometer  having  a 100- um  aperture  (ref.  46) . Hie 
photograph  shown  in  Fig.  13  was  taken  with  a Kodachrome  25  color-slide 
film.  Hie  density  range  0-3D  of  the  slide  was  digitized  with  a 25  um 
aperture.  A 512-  by  512-element  region  of  each  digitized  image  was 
selected  for  restoration.  Hie  type  and  extent  of  the  blur  present  in 
each  photograph  was  determined  according  to  the  procedures  described  in 
Sec.  5.0  and  the  values  of  used  in  the  Wiener  filters  were 
estimated  as  described  in  Sec.  3.4.  We  effected  the 
transmittance-domain  restorations  by  exponentiating  the  digitized 
density  values  over  a 2-decade  (or  a 3-decade)  range,  calculating  the 
2-dimensional  FFT  (refs.  17,  45  and  47),  applying  the  Wiener  filter  and 
then  calculating  the  inverse  FFT.  We  performed  the  density-domain 
restoration  by  restoring  the  density  values  directly  and  then 
exponentiating . 

We  first  compare  density-  and  transmittance-domain  restorations 
of  a photograph  containing  a focus  blur  with  a small  (approximately  4%) 
amount  of  spatial  variance.  Hie  photograph  of  a sign  taken  with  the 
camera  out  of  focus  is  shown  in  Fig.  7(a).  Using  the  OTF  given  by  Eg. 
19  with  a blur-circle  diameter  of  0.675  ran  on  the  film  and  ♦f/4n  * 25 
dB,  we  restored  the  image  in  the  density  domain  (Fig.  7(b))  and  in  the 
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' transmittance  domain  (Fig.  7(c)).  Hie  central  256-  by  256-element  region 

of  each  image  is  shown,  enlarged,  in  Figs.  8-10. 

Compared  with  the  photograph  of  the  same  sign  taken  with  the 

I 

camera  in  focus  (Figs.  7(d)  and  11),  both  restored  images  have  a 
resolution  close  to  the  limit  imposed  by  the  digitization.  Text 
completely  illegible  in  the  original  can  be  read  in  both  restorations. 
Hie  transmittance  restoration,  however,  may  be  slightly  preferable  to 
the  density  restoration.  In  particular,  the  text  is  more  legible  and 
there  is  less  ringing  where  the  dark  border  of  the  sign  joins  the  light 
wall.  A possible  explanation  is  that,  for  this  image,  the  logarithmic 
density  function  is  not  approximately  linear  over  the  extent  of  the  PSF 
of  the  Wiener  filter  (Sec.  3.2)  at  the  sharp  changes  in  exposure  (i.e. 
the  white  letters  against  the  black  background  and  the  black  sign 
against  the  white  wall) . Hiis  nonlinearity  only  occurs  when  the  image 
is  restored  in  the  density  domain. 


As  a second  example  of  the  removal  of  a spatially  invariant 
blur,  a sign  was  photographed  from  a moving  vehicle  to  produce  the 
motion-blurred  image  shown  in  Fig.  12(a).  Hie  blurred  transmittance 
image  was  restored  assuming  the  OTF  given  by  Eq.  20  (with  a blur  length 
of  1.43  cm  on  the  film  and  0 = 0°)  taking  4f/*n  = 30  dB.  Hie  result  is 
given  in  Fig.  12(b).  Despite  its  large  extent  (greater  than  1/4  of  the 
picture  width) , the  motion  blurring  of  the  sign  has  been  almost 
completely  removed  by  the  restoration. 


In  the  third  exanple,  the  original  photograph  (Fig.  13(a)) 
consists  of  2 areas:  one  contains  no  degradation,  the  other  contains 


FIGURE  8 Enlargement 


Enlargement  showir 
photograph  (Fig.  7(c 


the  central  region  of  a blurred 
taken  with  the  camera  out  of  focus 
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FIGURE  9 Transmission-domain  restoration  of  the  blurred  image  shown 
in  Fig.  8 
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FIGURE  10  Density-domain  restoration  of  the  blurred  imaqe  shown  in 
Fig.  8 
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FIGURE  II  ihe  region  of  the  sign  given  in  Fig.  8 photographed  with  the 
camera  in  focus 
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FIGURE  12  Restoration  of  a photograph  containing  spatially  invariant 
motion  blur.  The  original  blurred  image  (a)  is  shown  after 
restoration  in  (b) . 
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FIGURE  13 


The  background  region  of  the  photograph  (a)  is  degraded  by 
linear-motion  blur.  The  result  of  restoring  the  complete 
photograph  to  deblur  the  background  is  given  in  (b) . 
Enlargements  showing  a region  of  the  background  before  and 
after  restoration  are  given  in  (c)  and  (d) . Artifacts  due 
to  restoration  of  the  unblurred  portion  of  the  image  and  to 
film  saturation  are  apparent  in  (b) . 
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linear-motion  blur.  The  carer a tracked  the  moving  vehicle  so  the  image 

9 

t 

of  the  vehicle  was  correctly  recorded  while  the  image  of  the  stationary 
background  was  blurred.  The  result  of  restoring  the  complete  image  to 
remove  the  blurring  from  the  background  is  shown  in  Fig.  13(b).  We 
restored  the  image  in  the  transmittance  domain  using  the  OTF  given  by 
Eq.  20  with  a blur  length  of  0.61  mm  on  the  film,  0 = 0°  and  $-/*  = 20 
dB.  Enlargements  showing  a section  of  the  background  region  before  and 
after  restoration  (Figs.  13(c)  and  13(d))  illustrate  that  the  blurring 
has  been  effectively  removed.  For  example,  features  which  appear  as 
streaks  in  the  original  image  can  readily  be  identified  as  small  rocks 
after  restoration,  and  boundaries  between  light  and  dark  areas  have  been 
sharpened.  (The  background  consists  of  an  almost-vertical  bank  of 
earth.) 


Two  types  of  artifacts  are  evident  in  the  restored  image.  Both 
appear  as  severe  ringing  in  which  the  PSF  of  the  Wiener  filter  is 
displayed.  (Compare  the  ringing  with  the  PSF  of  the  Wiener  filter  given 
in  Fig.  3(e).)  The  first  type  of  artifact  is  caused  by  application  of 
the  Wiener  filter  to  the  unblurred  image  of  the  vehicle.  The  second  is 
due  to  film  saturation  in  2 over-exposed  areas  of  the  image  (in 
particular  in  the  wedge-shaped  feature  close  to  the  center  of  Fig.  13(a) 
and  in  the  area  in  the  upper-right  corner).  One  can  reduce  or  eliminate 
these  artifacts  by  editing  the  image  before  restoration  to  remove  the 
over-exposed  areas  (i.e.  by  matching  the  average  gray  level  of  the  area 
to  that  of  the  surrounding  region) , and  by  confining  the  restoration  to 
the  motion-blurred  portion  of  the  image  (i.e.  by  selecting  a region 
which  does  not  include  the  vehicle) . 
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The  fourth  image  to  be  restored  (Fig.  14(a))  is  degraded  by 
spatially  variant  motion  blur.  The  scene,  which  was  photographed  from  a 
moving  vehicle,  contained  objects  located  at  different  ranges  from  the 
camera.  In  addition,  the  camera  was  not  moving  in  the  olane 
perpendicular  to  the  optical  axis.  Both  the  length  and  direction  of  the 
blur  therefore  varied  throughout  the  image.  The  result  of  restoring  the 
front  of  the  building  (usirv  a blur  length  of  1.18  im  on  the  film,  0 = 
37.5°  and  = 20  dB)  is  given  in  Fig.  14(b).  Enlargements  showing 
the  regions  of  the  sign  and  the  door  before  and  after  restoration  are 
given  in  Fig.  15.  Comparison  with  the  unblurred  photograph  of  the 
building  (Fig.  14(c))  obviously  shows  that  text  which  is  illegible  in 
the  original  blurred  image  has  been  correctly  restored.  For  example, 
after  restoration,  the  street  number  ”382"  on  the  door  can  be  read  in 
Fig.  15(d)  as  can  the  telephone  number  "843-6652"  in  Fig.  15(c). 

Other  objects  in  the  photograph,  e.g.  the  car  to  the  left  of 
the  building  and  the  chimney  on  the  building,  are  located  further  from 
the  camera  and  are  not  correctly  restored.  The  artifacts  (PSF  ringing) 
generated  by  incorrect  restoration  of  these  objects  do  not  extend 
significantly  into  the  correctly  restored  regions.  Ringing  of  2 
highlights  on  the  car  vhere  the  film  has  saturated  is  also  evident  in 
the  restored  image. 


FIGURE  14  Restoration  of  a photograph  containing  spatially  variant 
motion  blur.  The  original  blurred  image  (a)  is  shown  in  (b) 
after  restoration  of  the  front  of  the  building.  Other 
regions  of  the  image  are  incorrectly  restored.  An  unblurred 
photograph  of  the  building  is  given  in  (c) . 
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Enlargements  of  the  images  given  in  Figs.  14(a)  and  (b) 
showing  the  region  of  the  sign  (a)  before  and  (c)  after 
restoration,  and  the  region  of  the  door  (b)  before  and  (d) 
after  restoration 


FIGURE  15 
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5.0  DETERMINATION  OF  TfE  OTF 


5.1  Interactive  Method 


To  restore  a blurred  image  with  a Wiener  filter,  one  must  know 
the  OTF  as  given  by  the  type  of  blur  (e.g.  out-of-focus)  and  its  extent 
(e.g.  diameter  of  blur  circle).  Hie  present  section  shows  how  this  can 
be  determined  interactively.  A second  method  for  determining  the 
required  information  which  consists  in  analysing  the  power  spectrum  of 
the  blurred  image  and  which  can  be  applied  for  certain  types  of  OTFs,  is 
described  in  Sec.  5.2. 

In  the  interactive  method,  a small  region  of  the  complete  image 
is  selected  and  restored  using  guesses  for  the  type  and  extent  of  the 
blur.  Hie  initial  guesses  are  improved  (usually  by  systematically 
repeating  the  restoration  as  a function  of  a single  parameter  such  as 
blur-circle  diameter)  and  the  values  which  subjectively  yield  the  best 
result  are  used  to  restore  the  complete  image. 

The  initial  guesses  for  the  type  and  extent  of  the  blur  can  be 
based  both  on  a priori  knowledge  and  on  direct  analysis  of  the  blurred 
image.  For  example,  in  restoring  the  image  of  Fig.  13  we  assumed  a 
priori  that  the  background  was  blurred  by  linear-motion  degradation. 
(Hie  moving  vehicle  was  not  blurred,  which  indicated  correct  tracking  of 
the  camera.  Over  the  1/250  s exposure  time,  the  camera  motion  relative 
to  the  stationary  background  is  then  approximately  linear.)  We  obtained 
initial  estimates  for  the  length  and  direction  of  the  motion  blur  by 
measuring  the  extent  of  the  blurring  of  objects  in  the  background  that 
were  assixned  to  contain  point  or  edge  structure  (ref.  5). 


UNCLASSIFIED 

42 


An  example  showing  variation  of  a parameter  from  the  initial 
estimate  to  optimize  the  restoration  is  given  in  Fig.  16.  Here,  we 
restored  a region  of  the  out-of-focus  photograph  of  a sign  shown  in  Fig. 
7(a)  using  8 different  assumed  values  for  the  diameter  of  the  blur 
circle  on  the  negative.  The  values  varied  in  0.025  nrn  steps  from  0.575 
mm  to  0.825  mm.  It  is  evident  that  the  best  restoration  is  obtained  for 
a blur  circle  diameter  of  0.675  mm.  This  value  was  used  to  restore  the 
complete  image  as  shown  in  Figs.  7(b),  7(c),  9 and  10. 


According  to  Eg.  15,  one  may  express  the  blur  in  the 
spatial-frequency  domain  by  multiplying  the  discrete  Fourier  transform 
of  the  ideal  image  by  the  OTF.  The  WIT1  of  certain  degradations  can 
consist  of  a well-defined  pattern.  For  example,  the  HIF  of  the  focus 
blur  given  in  Fig.  6(a)  contains  characteristic  ring  structure  while  the 
MTF  of  the  linear  motion  blur  given  in  Fig.  6(b)  contains  characteristic 
linear  structure.  Provided  that  it  is  not  obscured  by  the  noise  term  in 
Eq.  15  or  by  structure  in  the  discrete  Fourier  transform  of  the  ideal 
image,  the  WTF  of  the  blur  will  appear  in  the  power  spec  trim  of  the 
degraded  image.  Gennery  (ref.  13)  has  shown  that  in  some  cases 
(particularly  if  the  KTF  of  the  blur  has  zeros)  this  pattern  can 
identify  both  the  type  and  the  extent  of  the  blur. 


As  an  example  of  this,  four  regions  of  the  out-of-focus 
p>hotograp>h  of  the  sign  shown  in  Fig.  7(a)  were  selected  (Fig.  17)  and 
the  pover  spectrum  (i.e.  the  modulus  of  the  FFT)  of  each  was  calculated 


(Fig.  18).  The  ring  structure  present  in  each  power  spectrum  is 


anadien  de  b anadien  de  b anadien  e>f  p 

TIONNEMENT  TIONN'EMENT  TIONNEMENT 


lU'o  ' *1 


|(irr  » 41 


I l»r  »•  * -»  r<-*  u 

<n  •»>  mi 


Utnu  •>*71411  11  C /A*.  &U«|u!lii«  *01411  il  OAt  £>  li»  I U I ) > 'I  • Akll  II  1/1.  t>  ii» 

*»  11  *«<l  «*M  I il  1)  I"! I .■*»♦  I ,i  11  in  '»»« 


> I > • ' 1 * |i  , p » ♦ | - * • ' * » 

\nae>::n  l>:  i*|  v.nad  : d:  i liMti  'N  DJ  1* 

TIONNEMENT  | TIONNEMENT  | TIONNEMENT 

4 | f # ^ • '•••*♦  | 

M# 

‘ ' | *4  1 » > 1 * ••  W ».  4*  >•>  | 

.1  it  ».♦!  .*•> 


■ Ill  1 • -lit  II  - ‘ k- 

^ A A • % *« 


. | 1 < 

4 , \ « 1 • t •■« 


FIGURE  16  Determination  of  the  CRT  of  the  blur  interactively.  A 
region  of  a blurred  photograph  is  shown  in  the  upper  left. 
Assuning  the  blur  is  due  to  an  out-of-focus  camera,  the 
image  was  restored  using  8 different  assumed  values  of 
blur -circle  diameter.  These  values,  which  are  written  in 
the  lower  left  of  each  image,  varied  in  0.025  ran  steps  from 
0.575  mm  to  0.750  ran.  The  restoration  is  clearly  optimum 
for  a blur-circle  diameter  of  0.675  ran. 
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FIGURE  17  Four  regions  of  the  photograph  of  a sign  taken  with  the 
camera  out  of  focus  are  shown  in  (a) , (b) , (c)  and  (d) 


FIGURE  18  Identification  of  the  MTF  of  the  blur  by  oower  spectrun 
analysis.  The  power  spectra  of  4 regions  of  a photograph 
taken  with  an  out-of-focus  camera  (i.e.  the  4 images  given 
in  Fig.  17)  are  shown  in  (a),  (b) , (c)  and  (d) . While  each 
of  the  4 images  before  blurring  contains  different  spatial 
features  the  same  ring  structure,  which  is  characteristic  of 
the  out-of-focus  blurring,  is  evident  in  all  power  spectra. 
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char acteri Stic  of  the  focus  degradation.  Following  Eq.  19,  the  OTF  of 

the  focus  degradation  is  a Bessel  fwction  of  the  first  kind  and  first 

2 2 1/2 

order  with  argunent  w=(u  +v  ) ' divided  by  w.  This  function  is 
circularly  symmetric  about  the  frequency  origin  with  the  first  4 zeros 
located  at  Wj*l. 220/d,  w2=2. 234/d,  w^=3. 238/d  and  w4*4. 242/d,  where  d is 
the  diameter  of  the  blur  circle.  Hie  zeros  in  the  MIF  occur  at  the  same 
locations.  By  summing  the  power  in  ring-shaped  regions  about  the 
origin,  dividing  by  the  area  of  each  region  and  plotting  this  as  a 
function  of  distance  from  the  spatial-frequency  origin  (Fig.  19),  one 
can  determine  the  locations  of  the  first  3 minima  in  each  power  spectrum 
(corresponding  to  the  first  3 zeros  in  the  Bessel-f unction  OTF).  Each 
minimum  in  the  radial  power  distributions  appearing  in  Fig.  19  gives  an 
estimate  for  the  diameter  of  the  blur  circle.  Averaging  the  3 values 
obtained  from  each  graph  gives  the  following  estimates  for  the  average 
blur-circle  diameters:  region  (a),  d*0.67  mm;  region  (b) , d=0.66  mm; 
region  (c) , d-0.69  mm;  region  (d) , d*0.69  nm. 

This  method  can  be  very  accurate.  Here  it  was  able  bo  detect 
the  fact  that  the  upper  portion  of  the  sign  (regions  (a)  and  (to) ) was  4% 
less  out-of-focus  than  was  the  bottom  portion  (reqions  (c)  and  (d) ) . 
The  photograph  was  taken  with  the  camera  focused  at  a point  behind  the 
sign  and,  as  is  evident  in  Fig.  7(a),  the  lower  portion  of  the  sign  was 
closer  to  the  camera  than  the  upper  portion. 


Before  calculating  the  above  power  spectra,  we  high-pass 
filtered  each  image  using  a digital  approximation  to  the  two-dimensional 
Laplacian  operator  (refs.  5 and  13): 
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FIGURE  19  Hie  radial  power  distributions  of  the  4 Dower  spectra  given 
in  Fig.  18.  The  oscillation  present  in  each  distribution  is 
characteristic  of  the  focus  blurring  and  each  minimum  gives 
an  estimate  for  the  diameter  of  the  blur  circle.  Hie  graphs 
(a) , (b) , (c)  and  (d)  respectively  yield  average  values  of 
0.67  mm,  0.66  rrm,  0.69  mm  and  0.69  mm  for  the  diameter  of 
the  blur  circle  on  the  film  negative. 
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g’(x,y)  = g(x-l,y)  + g(x+l,y)  + g(x,y-l)  + g(x,y+l)  - 4g(x,y)  (22) 

This  filter  enhances  the  low-amplitude  higher  spatial  frequencies 
2 2 

approximately  as  u +v  , which  makes  the  MTF  pattern  more  easily  visible 
in  the  power  spectrum. 

When  the  power  spectrum  is  estimated  as  the  modulus  of  the  FFT, 
a discontinuity  effectively  exists  if  the  opposite  edges  of  the  image  do 
not  match  one  another  (refs.  15,  26  and  36).  This  discontinuity 

introduces  spatial- frequency  components  along  the  vertical  and 
horizontal  axes  of  the  power  spectrum  which  are  not  multiplied  by  the 
MTF  (ref.  13).  For  example,  the  vertical  structure  passing  through  the 
spatial-frequency  origin  in  the  4 power  spectra  given  in  Fig.  18  is 
caused  by  mismatch  between  the  upper  and  the  lower  edges  in  each  of  the 
corresponding  images  shown  in  Fig.  17.  Eliminating  these  undesired 
axial  spatial  frequencies  before  calculating  the  radial  power 
distributions  of  Fig.  19  (by  setting  the  appropriate  regions  of  each 
power  spectrum  to  zero)  significantly  improved  the  accuracy  with  which 
the  positions  of  the  minima  could  be  determined. 

In  general , the  power  spectra  of  different  regions  of  the  ideal 
image  will  have  different  structure  but  all  will  be  multiplied  by  the 
same  MTF  (assuning  a spatially  invariant  degradation) . Then  averaging 
the  power  spectra  of  different  regions  of  the  image  can  enhance  the  MTF 
pattern.  Averaging  the  4 power  spectra  of  Fig.  18  gave  the  enhanced  MIF 
pattern  shown  in  Fig.  20(a). 
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FIGURE  20  Enhancement  of  the  MTF  pattern  in  the  power  soectrun  of  a 
blurred  photograph  by  2 methods.  In  (a)  the  enhancement  was 
obtained  by  averaging  the  power  spectra  of  4 different 
regions  of  the  blurred  image.  Further  enhancement  was 
obtained  in  (b)  by  band-pass  filtering  the  averaged  power 
spectrum  (a) . The  power  spectrum  of  (a)  is  shown  before 
filtering  in  (e)  and  after  filtering  in  (f) . Hie  radial 
power  distributions  of  (a)  and  (b)  are  given  in  (c)  and  (d) 
respectively. 


IT" 


UNCLASSIFIED 

50 

, If  the  approximate  nature  of  the  blur  is  known,  then  a spatial 

filter  can  be  applied  to  the  powsr  spectrun  of  the  degraded  image  to 
further  enhance  the  MTF  pattern.  However,  the  filter  must  be  chosen  so 
that  it  does  not  distort  the  desired  MTF  pattern  or  introduce  additional 
structure.  The  power  spectrum  of  the  averaged  power  spectrum  given  in 
Fig.  20(a)  is  shown  in  Fig.  20(c).  It  is  evident  that  the  energy 
corresponding  to  the  MTF  of  the  present  blur  is  concentrated  within  a 
narrow  annular  range  about  the  spatial-frequency  origin.  This 
corresponds  to  the  fundamental  frequency  of  the  Bessel  function. 
Band-pass  filtering  the  power  spectrum  given  in  Fig.  20(a)  so  as  to 
retain  only  these  spatial  frequencies  considerably  enhances  the  MTF 

K 

pattern,  as  shown  in  Fig.  20(b).  The  power  spectrum  of  the  power 
spectrum  after  band-pass  filtering  is  given  in  Fig.  20(f).  Only  3 
minima  are  present  in  the  radial  power  distribution  before  filtering 
(Fig.  20(c))  vhile  after  filtering  a fourth  minimum  is  apparent  (Fig. 
20(d)).  Before  filtering  the  minima  in  the  power  spectrum  yield  an 
average  blur-circle  diameter  d=0.68  mm  while  after  filtering  the  value 
d=0.72  mm  is  obtained.  The  filtering  has  therefore  only  slightly 
altered  the  positions  of  the  minima. 

We  conclude  this  section  with  an  example  Which  summarizes  the 
use  of  the  above  procedures  to  restore  the  blurred  photograph  shown  in 
Fig.  21(a).  The  central  256-  by  256-element  region  of  the  digitized 
photograph  was  selected,  a Laplacian  high-pass  filter  applied  and  the 
power  spectrum  calculated  (Fig.  21(b)).  The  ring  structure  in  the  power 
spectrum  identifies  the  blur  as  being  due  to  an  out-of-focus  camera 
(with  circular  aperture),  and  the  first  3 minima  yield  estimates  of  1.72 
mm,  1.74  mm  and  1.77  mm  for  the  diameter  of  the  blur  circle  on  the 


FIGURE  21  Summary  of  the  procedure  used  to  restore  the  blurred  image 
(a) . The  power  spectrun  of  the  image  (b)  contains  ring 
structure  v*iich  identifies  the  blurring  as  being  due  to  an 
out-of-focus  camera  with  a blur-circle  diameter  of  1.76  nm 
on  the  film  negative.  Assuming  this  OTF,  the  image  was 
restored  using  a Wiener  filter  as  shown  in  (c) . The  same 
insignia  photographed  with  the  camera  in  focus  (d)  verifies 
the  correct  restoration  of  details  not  resolved  in  the 
original  blurred  image. 
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negative.  Measurement  of  the  RMS  noise  in  uniformly  exposed  regions  of 
the  photograph  indicated  that  a picture  element  with  transmittance  0.5 
would  have  a signal-to-noise  ratio  of  23  dB.  Using  a *f/4>n  equal  to  25 
dB  and  assuming  a blur -circle  diameter  of  1.75  mm,  we  restored  the  image 
in  the  transmittance  domain  with  the  OTF  given  by  Eq.  19.  The  result  is 
shown  in  Fig.  21(c).  The  photograph  of  the  same  insignia  taken  with  the 
camera  in  focus  (Fig.  21(d))  verifies  the  correct  restoration  of  details 
not  evident  in  the  original  blurred  image  (e.g.  the  beads  in  the  crown 
and  structure  within  the  central  circle) . The  resolution  (size  of  the 
smallest  detail  that  can  be  resolved)  has  been  increased  by  a factor  of 
5 by  the  restoration. 
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6.0  CONCLUSIONS 

Because  of  excessive  computing  times,  it  is  not  practical  to 
use  many  of  the  existing  digital  restoration  procedures  to  deblur 
high-resolution  images.  Wiener  filtering,  implemented  in  the 
spatial-frequency  domain  that  uses  an  FFT  algorithm,  is  one  approach 
which  can  be  applied  to  restore  high-resolution  images  (e.g.  images 
containing  512  by  512  elements)  and  which  uses  only  moderate 
computational  facilities. 

A complete  Wiener-filtering  restoration  procedure,  including  2 
methods  for  identifying  the  blur  that  analyse  the  blurred  image  directly 
using  only  limited  a-priori  knowledge,  was  described  here.  Examples 
showing  the  restoration  of  experimental  focus  and  motion  blurs  were 
given.  Hie  blurs  were  introduced  directly  in  the  camera  and  normal 
"amateur"  photographic  films  and  cameras  were  used. 

Correct  application  of  the  Wiener  filter,  and  in  particular  a 
knowledge  of  its  limitations,  is  fundamental  to  obtaining  an  acceptable 
restoration.  These  limitations  were  described  in  detail  with  regard  to 
restoration  of  blurred  photographs.  Techniques  for  restoring  blurred 
photographs  vrtiich  take  account  of  the  properties  of  the  Wiener  filter  as 
well  as  other  practical  considerations  (e.g.  film  nonlinearity,  artifact 
generation,  edge  effects  etc.)  were  described. 

The  most  ser ious  disadvantages  of  a Wiener  filter  are  that  it 
can  only  restore  blurs  containing  a small  amount  of  spatial  variance,  it 
does  not  simultaneously  assume  the  correct  models  for  film  noise  and  for 
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the  blurring,  and  it  does  not  make  use  of  known  physical  constraints 
(e.g.  the  ideal  inage  cannot  contain  negative  values  or  be  infinite). 
It  incorrectly  assumes  that  the  signal-to-noise  ratio  of  the  film  is 
independent  of  exposure.  Furthermore,  the  restoration  is  linear  (and 
therefore  sub-optimum  (ref.  1))  and  is  based  on  minimizing  the 
mean-square  error  between  the  ideal  and  the  restored  images.  It  is 
doubtful  whether  the  mean-square  error  criterion  correlates  well  with 
subjective  estimates  of  image  quality  (refs.  2 and  29).  (Many  of  the 
other  restoration  procedures  suffer  the  same  shortcomings  however.) 

The  present  report  has  demonstrated  that,  in  situations  of 
practical  interest,  significant  restorations  of  high-resolution  blurred 
photographs  can  be  obtained  through  the  proper  use  of  an  inverse  Wiener 
filter.  The  techniques  described  here  have  recently  been  applied  to  the 
more  difficult  problem  of  restoring  blurred  ohotoqraphs  taken  under  poor 
conditions.  Useful  improvements  in  resolution  have  been  obtained 
despite  incorrect  film  exposure  and  high  film  granularity  which  occurred 
in  severed  of  the  photographs. 

Methods  are  being  investigated  to  improve  the  subjective 
quality  of  the  Wiener-filter  restorations,  to  increase  the  range  of  OTFs 
that  can  be  restored  and  to  decrease  the  computing  times  required.  In 
particular.  Hunt  (ref.  2)  has  demonstrated  that  inclusion  of  the 
frequency  response  of  human  vision  in  the  Wiener  filter  produces 
improved  results.  Provision  to  specify  the  OTF  as  a 2-dimensional 
discrete  Fourier  transform  rather  than  only  as  an  analytic  function 
would  allow  blurs  with  arbitrary  PSFs  to  be  restored.  The  oresent 
restoration  procedure  will  be  implemented  on  a PDP  11/40  minicomputer 
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interfaced  to  an  array  processor  with  a large  data  memory.  Compared  to 
the  present  software  implementations,  the  array  processor  will  decrease 
the  time  required  to  transform  and  filter  the  image  by  2 orders  of 
magnitude . 
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